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ABSTRACT 

The focus of this paper will be to argue for an 
effective and concise evaluation of special education programs 
serving bilingual or limited English proficient (LBP) students. 
Although Public Lav 94-142 specifies that all children be provided 
with free appropriate education according to their individual needs, 
ensuring compliance with these requirements has been problematic, 
particularly in the provision and evaluation of appropriate programs 
and services to bilingual children. The future success of special 
education programs serving LBP students will largely depend on the 
ability to evaluate and communicate the ultimate successes and 
failures of such programs. In addition, while defining the overall 
worth of these programs, exemplary services and programs can be 
identified for dissemination. Advocates of special education for LEP 
students oust engage in evaluation activity for the future of this 
population. Special educators must realize that evaluation is a 
requisite in the process of planning and delivering special education 
services to all populations. (Author) 



****** ****** *********************** ********************* *************** 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 



••t******************************************************************** 



ERIC 



fVATKMWUL MSTffUTI & &tfCAVtOH 

EDUCATIONAL RfSQUBCES WFOKfcUTlON 



* 




cento ami 



Mm ctwngn hm btm irw*e to **npw# 



m*M do net fi««WWNV «*paw»oJ critical MS 

ptmnon of po$*cv 



THE NEED TO EVALUATE 
SPECIAL EDUCATION PROGRAMS 
SEF7IN" BILINGUAL FTUDENTF 



by 



Juan C. Gonzalez 

.ilingual special Education 
Education Building ^Uo 
The University of Texas 
Austin f Texas 78712 



PERMISSION TO REPRODUCE THIS 
MATERIAL MAb BEEN GRANTED BY 



TO THE EDUCATIONAL RESOURCES 
HtfORMATTO* CENTER mmCV 




Paper presented at the Annual Meeting of the American Educational 
Research Association (68th, New Orleans, LA, April 25*27, 1984} 



ABSTRACT 



The Heed to Evaluate Special Education Programs Serving Bilingual Students 

The focus of this paper will toe to argue for an effective and concise 
evaluation of special education programs serving Mlingual or limited English 
proficient (LEF) students. Although Public Lav 9b-lU2 specifies that all 
children toe provided with free appropriate education according to their 
individual needs , ensuring compliance of these requirements has toeen proble- 
matic, particularly f in the provision and evaluation of appropriate programs 
and services to toilingual children. 

The future success of special education programs serving LEP students 
will to a large extent depend on the atility to evaluate and communicate the 
ultimate successes and failures of such programs. In addition , while de- 
fining the overall worth of these programs, exemplary services and programs 
?ar« be identified for dissemination. Advocates of special education for LEP 
students must engare in evaluation activity for the future of this population, 
fyec iul educators must realize? that evaluation is a requisite in the process 
cf planning and delivering special education services to all populations. 



THE NEED TO EVALUATE SPECIAL EDUCATION PROGRAMS SERVING BILINGUAL STUDENTS 



The focus of this paper will he to argue for an effective and concise 
evaluation of special education programs serving bilingual or limited English 
proficient (LEP) students. Presently, we do not have ouch detailed data 
about the state of the art of special education programming for LEP students. 
In addition the effects of national and state evaluation systems on programs 
and services provided to LEP population is unclear. This lack of informa- 
tion is delaying improvement and further development of programs. Decision 
makers need reliable and valid data that effective evaluation methodologies 
can provide. The absence of these data ostensibly affects both the quality 
arid type of special education programs offered to bilingual students. 
LEGISLATION 

T*e most significant federal legislation to target LEP persons for 
special education was the Public Law 9b-lk2. Many of the issues expressed 
in :-.L. ?l»-lL2 were a "Ir-.ct result of parents and special educators con- 
cerned about minority related issues such as inappropriate testing procedures, 
labeling, disproportionate numbers of culturally diverse children in special 
education categories and inadequate training of teachers. Some parts of 
F.L. 9'^- 1L2 requirements directly focused upon the improvements of services 
ar.i programs for culturally and linquistically different children. 

F.L. 9k-lUl (under section 6l8 (a)) specifically stipulated that the 
ir.iact of the program authorized under this act and the effectiveness of state 
,...j7, r ... tr , » irrur « t jj e free appropriate public education of all handicapped 
children be measured and evaluated. According to Ballard and Zettel (1977), 
L. r ' — 1-2 has several purposes: 

«A'y insure the availability of special education for children that 
require it, 
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(B) insure that fairness and appropriateness are inherent in the 
decision-making process , 

(C) establish clear management and auditing procedures, and 

(£>) provide financial assistance to the states and local school dis- 
tricts* 

In order to evaluate the impact of F.L. ?l*-ll*2, each state is required 
to keep records, such as the number of students participating in special 
education programs, their achievements, and the number of handicapped children 
vho need and are not receiving a free appropriate public education* Using 
this information as veil as other data, the Department of Education is re- 
quired on an annual basis, to report the impact of P.L. 9U-IU2 to the Congress, 
The department must address the folloving areas: (a) effectiveness of proce- 
dures providing handicapped children with educational services in the least 
restrictive environment, (b) effectiveness of procedures for improving instruc- 
tional programs for children in day or residential facilities, and (c) effec- 
tiveness of procedures that would eliminate or reduce the erroneous classifi- 
cation of children. In order to carry out these analyses and evaluations, 
the Department is directed to use statistically valid survey techniques and 
tr, rahmit the data collected to the National Center for Educational Statistics. 

Although F,L. 9^-1^2 specifies that all children be provided with free 
appropriate education according to their individual needs, ensuring compliance 
cf these requirements has proven problematic (Altschuld * Dovnhover, 1980), 
particularly in the provision and evaluation of appropriate programs and 
r*?rvices to bilingual children. One of the reasons is that the identification, 
ac cessment and placement procedures used by states are relatively crude. 
Also, the system used tc aggregate data at the state and national levels is not 
diffidently veil organized to identify all of the culturally different pop- 



ulations. lndeed t according to Ortiz and Yates (1981 ) t the data indicating 
incidence rates for Hispanics has proven highly subject to questions of 
reliability and concurrent validity relative to expected state and national 
norms. Finally, since the focus on special education programs" and services 
is a relatively new approach to meeting the needs of bilingual students t 
only a very few states have developed evaluation models and methodologies 
to t 1udge the merits of these highly specialized programs. In summary, the 
state of the art of special education programming for bilingual populations 
is simply too underdeveloped to have fully functional systems of assessment, 
program development and evaluation in place, 

THE KEFD FOB SPECIAL EDUCATIOH FOR BILINGUAL STUDENTS 
The LEr population 

Th-* number of LEP persons in the United States has become a major issue 
in justifying programs, passing legislation, and obtaining funds. The 
sponsors of F.L. 9^*1^2 recognized that a substantial portion of the LEP 
population vas not being served appropriately by special education programs* 
Tr.e I allure of special education to meet the needs of LEP students primarily 
f^sed on the language usage of the child. According to Gorzales and 
rtiz (1977), the following analysis describes the situation: 

In all cases, when the question arose as to the cause of these failures 
or* the part of the Spanish-speaking child, language emerges as the prime 
reason for their failure in the school system. Current educational 
iraetices insist that all children function in one set of language 
:.fcxllr f and yet nothing has been done to accomodate the hilingual child 
vith inadequate English skills. Instead, educators often assume the 
child will pick up the necessary skills by osmosis. As a result, many 
cf these children have fallen into a pattern of failure and eventually 



are referred to special education* This common practice contributed 
to legitimizing the removal of the culturally and linguistically differ- 
ent child from the mainstream of education (p.33M. 
As this citation indicate? , language is often a significant factor in 
the educational process of LEP students. The number of persons whose 
dominant language is not English is quite significant* There are at least 
26 million persons in the United States f including about 5 million school- 
aged children, vhose dominant language is not English. The great majority 
were bom in the United States and are citizens. More specifically, about 
two thirds of these persons and more than four fifths of the school-aged 
children are native born (U.S. Bureau of the Census, 1976, 1980). Other 
findings indicate that 1 person in 8 in the United States has a non-English 
language background. Furthermore, the specific language background of more 
than a third of all language minority persons and 60 percent of all school- 
aged children in this group was Spanish. Persons vith Spanish language back- 
ground numbered 13.2 million in I960, and their numbers are incrersing relative 
to all ether population subgroups. Language minority persons vere located 
in every state; however, 3 out of 5 were located in five states: Arizona, 
Texas, California, Nev Mexico, and Colorado. These states, plus Rev York, 
Florida, Illinois, and Nev Jersey account for 90 percent of the Spanish language 
background population. 

The above data provide, for the first time, individual states' estimates 
r r current language usage as well as language backgrounds of school-aged 
children. It is important that this type of data be collected, since statistics 
and statistical analyses have become a driving force in social and educational 
policy making. According to the U.S. Department of Labor (1980), this type of 
data is crucial for th? following reasons: What problems ve address and hov 



ve address the» are frequently determined by the availability of numbers and 
their translation into terms relevant to policy issues and policymakers. 
The unemployment rate is perhaps the most closely watched of all social indi- 
cators, determining where and how resources will be used. It is, then, an 
issue or fundamental importance that data and analyses are limited concerning 
the Hispanic population of our Ration and its particular needs. The dearth 
of information may, indeed, permit "benign neglect H in seme cases and mis* 
directed action in others (p.l). As the quote points out, a paucity of data 
concerning LEF students is of vital importance in better serving this pop- 
ulation in special education. The process of evaluation may well be the 
process that is best suited for the collection of valid and reliable data 
for decision-makers* 
MEETING THEIR NEEDS 

The failure of special education to meet the needs of LEP students f 
given historical data indicating indiscriminate labeling and placement, 
necessitates a full commitment on the part of special educators to better 
n^rvf this population. A review of professional literature indicates that 
minority children are over-represented in the classes for th*' handicapped, 
j-nr? icrularly those classes for the mentally retarded {Dunn* 1968; Mercer, 
19?t). Contrary to this literature, recent research by Ortiz and Yates (I98l) 
indicates that Hispanics in Texas are under-represented in all categories of 
handicapping conditions with the exception of learning disabilities. In this 
category, Hispanic students are over-represented by more that 300 percent. 
Of interest is that 80 percent of Hispanic students eligible for services are 
jriacel in either speech therapy or LD programs. Both of these categories 
ar* 1 hti£ uage-related and suggest that educators may be unable to distinguish 



linguistic differences from real deficits* Traditional assessment and place- 
ment procedures used by special education personnel have resulted in errors 
in the placement of LEP students into special education. Practitioners are 
in need of proven procedures that ensure non-biased assessment and placement 
for LEP students. 

One of the most encouraging developments in meeting the need of LEP 
students are the regulations under P.L. 9^-lte. The stren"*tb of P.L. 9k-lU2 
rests in its specific definition of special education t as veil as in its 
detailed recommendations regarding the types of services required to meet the 
legislated objectives. The act called for specific instructional and cur- 
ricular programs associated vith the training of teacher personnel and the 
development of "individualized education programs" (Karlit2 t 1982). Of 
I articular significance to language minority populations was the inclusion 
into the legislation of the "native language" descriptor. By including this 
descriptor, P.L. 9^-1^2 was effectively linked to the Bilingual Education Act. 
This laid the bases for requiring that bilingual^LEP children be assessed in 
M,'.-ir } rimary language. Functionally, this meant that Eilingual-LEP students 
with possible handicapping conditions must be assessed to determine their 
native: language and further testing must be conducted in the child's primary 
Inr-^age. Assessment of any child with possible handicapping conditions can 
be difficult tank; however, it is particularly difficult to assess a bi- 
lingual -LET child when one considers the few trained bilingual assessment 
specialist and few assessment instruments specifically developed for this 
turv*'t y.ot ulation. 

Although the regulations under P.L. 9^-1^2 have been in existence for 
r^veraJl years , much of what has been done in this area has gone unnoticed. 
This ir due to a large measure to limited research, development, dissemination, 
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and evaluation* These limitations have caused a restricted rate of develop- 
ment and expansion of special education programs serving LEP populations. 
From the arguments developed above, it appears that the needs of LEP students 
in special education programs are circumscribed by four major issues: 
(a> appropriate and accurate assessment procedures (Mendoza, 1983) (to) develop- 
ment across states of systematic special education delivery systems t (c) research 
and evaluation of programs models serving LEP populations and (d) dissemination 
of exemplary programs and practices* Each of these problem areas represents 
an enormous undertaking for professional educators in coming years. 
EVALUATION ISSUES 

Research has begun to identify what appears to be necessary elements 
for high-quality special education programs serving LEF exceptional children. 
For example f Milne (1982) outlined five components that are critical to 
serving exceptional LEF students* These components are assessment procedures, 
cultural awareness training of monolingual teachers f curriculum and methodology t 
learning styles, and parental involvement. The relative newness of program 
delivery to this exceptional population demands flexibility on the part of 
special educators, and administrators. Although there is no single "best" 
jr vraj:. design, it should be feasible to draw on the successful experiences 
of a number of programs. It falls, then, on the evaluation process to assist 
pr';rram funders, administrators, teachers, students, and parents to better 
understand the complexities of these new programs and services. Through 
hifh-quality evaluations, scarce resources may best be put to use to ensure 
th^ noi:t appropriate service delivery continuum at local levels. 

House (1976) states that the process of evaluation is a tool in deciding 
who rets what: 

Evaluation is by its nature a political activity. It serves decision- 



makers , results in reallocation of resources and legitimizes who gets 
vhat. It is intimately implicated in the distribution of basic goods 
in society. It is more that a statement of ideas « it is a social 
mechanism for distribution, one which aspires to institutional status* 
(p.76) 

Bilingual education programs and more recently special education programs 
serving bilingual students can be classified as innovative and nontraditional. 
Because of the recency of these types of programs t these programs have pre- 
sented e valuators vith practical and theoretical considerations previously 
unencountered. According to Gonzales and Baumanis ( 198l ) f evaluators have 
been faced vith the following questions: 

1. Kov can a practical evaluation be operational! zed on a nationwide 
basis? 

2. How can we measure the success of a specialized academic-social 
change program that may well be influenced by variables hitherto 
unmeasured - or perhaps unidentified? 

The persistence of evaluators to use traditional objective-based 
evaluation approaches is largely due to federal and state policies requesting 
quantifiable* objective data. These policies have affected not only special 
education and bilingual education but almost every educational program re- 
ceiving federal funds. Gonzalez and Baumanis (1981) state that the traditional 
or behavioral objective approaches of evaluation are based on the assumption 
that all learning outcomes are measurable* i.e. quantifiable. The traditional 
model emphasizes student preformance, mastery ability, and aptitudes; progress 
is measured through utilization of specified behavioral objectives and the 
use of standardized tests. However* it has often been statea (Olroedo, 1977; 
Oakland, 1973) that testing of minority children has been linguistically, 
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culturally and ethnically biased. P.L. 9^-1^2 assured the use of the students 1 
native language when being tested and required the use of culturally appro- 
priate testing instruments. 

Although the literature relating to evaluation theory and research 
is much richer and more sophisticated today than it vas a decade ago, actual 
results of evaluation studies indicate that from an evaluation respective, 
many of the problems posed by the structural as veil as theoretical intri- 
cacies of the innovative programs remain unresolved. Even so, recent 
literature (Berke, 1983; Weiss, 1977) indicates that evaluations of innovative 
programs have significantly influenced nolicymaking. Federal regulations 
mandating accountability (i#e., P.L. 9 1 * **2) require evaluation to produce 
empirical evidence on program impact. . he evaluation data collected in res- 
pons V these mandates become the basis for far-reaching decisions regarding 
maintenance, institutionalization and expansion of successful programs, as 
veil as modification or elimination of unsuccessful ones. 

Special education programs aimed at bilingual students encounter 
problems: associated vith many other federally mandated efforts. It has most 
of the problems coimon to nonethnically oriented programs - specifically 
thoc*- relating to (l) diversity, obscurity and conflict of goals, and (2) the 
lack of clarity regarding assessment framework and proven interventions. 
Unlike other programs and very similar to the more established bilingual ed- 
ucation programs, special education for bilingual students adds a complex 
dimension to the other difficulties by introducing still another reality - 
teaching and evaluating in tvo languages. Language development and testing 
of LE- students have been the critical debates among bilingual and special 
e?ucatorr. ,ohen (i960) lists several relevant questions: 

I t Is a child's oral language development as important as (or more 
important than) reading development? 
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2* Should oral language skills be used to identify a child 1 s dominant 
language? 

3* Should an assessment of oral language be based on receptive language 

abilities or productive language abilities? 
k m Should reading skills development and oral language development be 

related in a program? 
j. Should a program emphasize oral language development in Spanish, oral 

language development in English, or both? 

0. Can there be parallel oral language development in two languages? 
(F.36)' 

Again, language with respect to special education plays a cri "cal 
part in serving LEP students and in the evaluation of these services • The 
following d r sources can be used in assessing successes of special education 
programs serving LE? studeuts. These aroas may potentially provide a basis 
for administrators and program evaluators who desire to initiate or improve 
rpecial education for LEP students: 

1. appropriate needs assessment 

c. appropriate referral of bilingual and LEP students to special education 

'% appropriate language assessment (dominance and proficiency) 

i t appropriate non-biased intelligence testing 

% appropriate involvement of staff in the development of IEP's 

C, appropriate and specific IEP's 

7. appropriate instruction and curriculum 

K appropriate entry-exit criteria 

9 # behavior of students, including attendance, learning rate and achieve- 
ment levels 

10, contextual changes, i.e. institutionalization, parent attitudes, 
teacher attitudes and so on 
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11. appropriate training and in-service of personnel, and 

12. program management and organization 

The number and complexity of the variables suggest an urgent need for the 
conceptual and technical development of evaluation procedures to assist 
funders , administrators, teachers, students and pjrents to better understand 
the complexities demanded of these programs and services. Although federal 
legislation has increased th* development of special education programs aimed 
at LEP students, there is no evaluation evidence that has focused on the 
operation of these programs, the instructional practices, or the effects on 
the students* 
rJ7J?£ DIRECTIONS 

As stated earlier, special education programs and services for all 
populations are to be evaluated as stipulated in P»L. 9^-l^S. Although 
this evaluation mandate has not focused on programs serving LEP students, it 
is likely to receive increased attention in future years. This is due to 
the increased number of LEP students that will be receiving special education 
lr. the- :?S0 f s and 1990 f s. Support for this contention also exists in the 
value placed on evaluation by special and general administrators at a time of 
m 2 lining enrollments and fiscal resources. Evaluation results can and most 
likely will provide the needed rationale to aid in making difficult decisions 
«ir/>ng program alternatives, as veil as suggesting* more cost efficient ways of 
conducting other programs. 

Another issue that may have a substantial impact on future use of 
♦•vilvition results is that of evaluation rethodology. In viev of the recency 
of rpecial education programs serving LEP students, it can be assumed that 
tho or user of evaluation will be oriented toward increasing understanding 
at out these programs rather than directly and solely influencing decision- 
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making. If this orientation is accepted, then the evaluation method most 
suited to that end is the qualitative or naturalistic method (Gonzalez ♦ 
Baumanis, 1981; Guha ♦ Lincoln, 198l; Patton f 198oh According to Tymitz 
(1983), the naturalistic approach to evaluation of bilingual special education 
programs is a favorable alternative (not a substitute) to the traditional 
quantitative methodology* 

The flexibility of the naturalistic inquiry contrasts markedly vith the 
traditional evaluation approach. The naturalistic approach attempts to arrive 
at an understanding rather than at providing an explanation or validation to 
prespecified theories or hypothesis. Stake (1975) describes his case study 
approach to e/aluation as follows: 

1. It orients more directly to program activities than program interest, 

2. it responds to audience requirements for information, and 

3. the different value-perspectives are referred to in reporting the 
success and failure of the program, (p. 8) 

According to Stake (19T5), a naturalistic approach stresses the complex, 
holistic, and multiple variables that affect ti*r programs. While the traditional 
approach dictates presentation of quantifiable data in formal written reports, 
thf' j>rer,entation of the naturalistic approach might assume a variety of forms, 
e.g., written reports, simulated programs interactions, films, taped activities, 

The aim of the report is to increase the understanding of what is really 
fcir.f on in the program. 

Tymitz (1983) gives three recommendations for future evaluations of 
r.T'-ifil education programs serving LEP students: 

First - given that program goals involve both process and product out- 
comes, the design of the evaluation strategies to collect and 
assess data must become more congruent with the phenomena under 
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study. It is critical that study designs have the capacity to 
evaluate the many different ways children learn and develop and 
that studies are able to capture and portray the perceptions 9 
attitudes 9 and beliefs of all program participants* 
Second - rather than a preoccupation with fault-finding t evaluation oust 
engage in the far more difficult task of understanding and des- 
cribing program effects in their entirety* 
Third - as bilingual special education continues to develop » evaluation 
designs must attend to the philosophical 9 applied research » and 
legal issues that confront practitioners, (p. 367) 
Although the mandate for inclusion of LEF populations in special education 
has existed since the 197 5 passage of F.L. 9^-1^2 f the systematic provisions 
of the programs and services has progressed slowly. A clear conception of the 
scope and severity of the problem can only be made possible through compre- 
hensive multiple-year evaluations of those programs and services at state and 
local levels. 
COUCLUSIOK 

The future success of special education programs serving LEF students 
vill to a large extent depend on the ability to evaluate and communicate the 
ultimate successes and failures of such programs* In addition, while de- 
fining the overall worth of these programs, exemplary services and programs 
can be identified for dissemination. Advocates of special education for LEP 
students must engage in evaluation activity for the future of this population, 
"pecial educators must realize that evaluation is a requisite in the process 
of planning and delivering special education services to all populations. 
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